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(57) Method for speech signal synthesis by means 
of time concatenation of waveforms representing ele- 
mentary units of speech signal, in which: at least the 
waveforms associated to voiced sounds are subdivided 
into a plurality of intervals, corresponding to the 
responses of the vocal duct to a series of excitation 
impulses of the vocal cords, synchronous with the fun- 
damental frequency of the signal; each inten^al is sub- 
jected to a weighting: the signals resulting from the 
weighting are replaced wrth a replica thereof shifted in 
time by an amount that depends on a prosodic informa- 
tion: and the synthesis is carried out by overlapping and 
adding the shifted signals. In each interval of original sig- 
nal to be reproduced in synthesis, an unchanging part is 
identified, which contains the fundamental information 
and which is reproduced unaltered in the synthesized 
signal, and the operations of weighting, overlapping and 
adding involve only the remaining part of the interval. 
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Description 

The invention desaibed herein relates to speech synthesis and more particularly to a synthesis method based on 
the concatenation of waveforms related to elementary speech units. Preferably, but not exclusively, the method is applied 

5 to text-to-speech synthesis. 

In these applications, a text to be transformed into a speech signal is first converted into a phonetic-prosodic rep- 
resentation, which indicates the sequence of con'esponding phonemes and the prosodic characteristics (duration, inten- 
sity, and fundamental period) associated to them. This representation is then converted into a digital synthetic speech 
signal starting from a vocabulary of said elementary units, which in the most common case are constituted of diphones 

10 (voice elements extending from the stationary part of a phoneme to the stationary part of the subsequent phoneme, the 
transition between phonemes included). For the Italian language, a vocabulary of about one thousand diphones ensures 
the phonetic coverage, allowing all admissible sounds for Italian language to be synthesized. 

In systems for text-to-speech synthesis, methods based on the concatenation, in the time domain, of the waveforms 
representing the various elementary units can be used for the generation of the speech signal: these methods are very 

15 flexible and guarantee good syndetic speech quality. 

An example is described by E. i\/Ioulines and F. Charpentier in the paper "Pitch-synchronous v^veform processing 
techniques for text-to-speech synthesis using diphones". Speech Communication, Vo). 9. No. 5/6. December 1 990. pages 
453-467. This method is based on the technique known as PSOLA (Pitch-Synchronous Overlap and Add), to apply the 
prosody imposed by the synthesis rules and concatenate the wave-forms of the elementary units. At least for the voiced 

20 segments of the original signal, the PSOLA technique carries out an analysis by applying a pitch-synchronous windo- 
wing, in particular by using Manning windows whose duration is roughly twice the fundamental period (pitch period), 
thereby generating a s^uence of partially overlapping short-term signals; in the synthesis phase, the signals resulting 
from the windowing are shifted in time synchronously with the fundamental period imposed by the prosodic rules for 
synthesis; finally, the synthetic signal is generate by overlapping and adding the shifted signals. To reduce conputational 

25 complexity, the second step can be carri^ out directly in the time domain. 

The complete windowing of the individual intervals of the original signal requires a relatively heavy computational 
load and moreover it constitutes an alteration of the original signal extending over the entire interval, so that the synthetic 
signal sounds less natural. 

According to the invention, a synthesis method is provide in which that part of each interval of the original signal 
30 which contains the fundamental information is left unchanged, and only the remaining part of the interval is altered: this 
way. not only processing time is reduce, but the natural sounding of the synthetic signal is also improved, since the 
main part of the interval is the exact r^roduction of the original signal. 

The invention therefore provides a method for the speech signal synthesis by means of time-concatenation of wave- 
forn^ r^resenting elementary speech signal units, in which: at least the waveforms associated to voiced sounds are 
35 divide into a plurality of intervals, con-e^nding to the responses of the vocal duct to a series of impulses exciting the 
vocal cords synchronously with the fundamental frequency of the signal; the waveform in each interval is weighted; the 
signals resulting from the weighting are replaced with a replica thereof, shifts in time by an amount depending on a 
prosodic information; and the synthesis is candied out by overlapping and adding the shifted signals; and in which: 

40 - a current interval of original signal to be reproduced in synthesis is subdivided into an unchanging part, which lies 
between the interval beginning and a left analysis edge r^resented by a zero crossing of the original speech signal 
that meets pre-determined conditions, and a changeable part, which ties between the left analysis ^ge and a right 
analysis edge essentially coinciding with the end of the current interval, the left and right analysis edges being 
associated, in the synthesized signal, respectively with a left and a right synthesis edge, of which the former coincides 

45 with the left analysis edge, with reference to a start-of-interval matter, and the latter coincides essentially with the 
end of the inten^al in the synthesized signal; 

- a first connecting function, which has a duration equal to that of the segment of synthesized waveform lying between 
the left and right synthesis edges and an amplitude which decreases progressively and is maximum in corespond- 
ence with the left analysis edge, is applied on the part of waveform on the right of the left analysis edge of the current 

so interval of original signal; 

- a second connecting function, which has a duration equal to that of the segment of synthesized waveform lying 
between the left and right synthesis edges and an amplitude which increases progressively and is maximum in 
correspondence with the beginning of said subsequent interval, is applied on the part of waveform on the left of the 
subsequent interval of original signal to be reproduced synthetically; 

55 - each interval of synthesized signal is built by reproducing unchanged the waveform in the unchanging part of the 
original interval and by joining thereto the waveform obtained by aligning in time and adding the two waveforms 
resulting from the application of the first and second connecting functions. 
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For the sake' of further clarification, reference is nfiade to the enclosed drawings, which illustrate an embodiment of 
the invention given by way of non-limiting example and where: 

Figure 1 is the general outline of the operations of a text-to-^eech synthesis system through concatenation of 
5 elementary acoustic units; 

- Figure 2 is a diagram of the synthesis method through concatenation of diphones and modification of the prosodic 
parameters in the time domain, according to the invention; 

- Figure 3 represents the waveform of a real diphone, with the markers for the phonetic and diphone borders and 
the pitch markers; 

10 - Figures 4. 5 and 6 are graphs representing how the prosodic parameters of a natural speech signal are modified 

in some particular cases, according to the invention; 

- Figures 7A. 7B. 8A, 8B, 9A, 9B. 10A and 108 are some real examples of application of the method according to 
the invention for the modification of the fundamental period on segments of the diphone in Figure 3; 

- Figures 11 - 18 are flow charts of the operations for determining the left analysis and synthesis edge. 

15 

Before describing the invention in detail, the structure of a text-to-speech synthesis system is briefly described. 
As can be seen in Figure 1 , as a first phase the written text is fed to a linguistic processing stage TL which transforms 

the written text into a pronounceable form and adds linguistic markings: transcription of abbreviations, numbers 

application of stress and grammatical classification rules, access to lexical information contained in a special vocabulary 

20 VL. The subsequent stage. TF, carles out the transcription from orthographic sequence to the corresponding string of 
phonetic symbols. On the basis of a set of prosodic rules RP, the prosodic processing stage TP provides duration and 
fundamental period (and thus also fundamental frequency) for each of the phonemes leaving TR This information is 
then provided to the pre-synthesis stage PS. which determines for each phoneme, the sequence of acoustic signals 
forming the phoneme (access to diphone data base VD) and, for each segment, how many and which intervals, with 

25 duration equal to the fundamental period, are to be used (in the case of voiced sounds) and the corresponding values 
of the fundamental period to be attributed in synthesis. These values are obtained by interpolating the values assigned 
in correspondence with the phoneme borders. In the case of unvoiced or "surd" sounds, in which there are no periodicity 
characteristics, the inten^als have a fixed duration. This information is finally used by the actual synthesizer SINT which 
performs the transformations required to generate the synthetic signal. 

30 Figure 2 illustrates in greater detail the operation of modules PS and SINT. The input is constituted by the current 
phoneme identifier F}. by the phoneme duration Dj and by the values of the fundamental period Pi.i at the beginning of 
the phoneme and Pj at the end of the phoneme, and by the identifiers of the previous phoneme Fj.^ and of the subsequent 
one Fj^i. The first operation to be performed is to decode diphones DF;.^ and DFj and to detect the markers of diphone 
beginning and end and of phoneme border. This information is drawn directly from the data base or vocabulary storing 

35 diphones as waveforms and the related border, voiced/unvoiced d^ision and pitch marking descriptors. The subs^uent 
module transforms said descriptors taking the phoneme as a reference. On the t>asis of this information, a rhythmic 
module computes the ratio between duration D, imposed by the rule and the intrinsic duration of the phoneme (memorized 
in the vocabulary and given by the sum of the two portions of the phoneme belonging to the two diphones DFj.^ and 
DFj). Then, taking into account the modification of the duration, it computes the number of intervals to be used in synthesis 

40 and determines the value of the fundamental period for each of them, by means of a taw of interpolation between value 
P|.1 and Pj. The value of the fundamental period is then actually used only for voiced sounds, while for unvoiced sounds, 
as stated above, intervals are consider^ to be of f ix^ duration. 

For the actual synthesis, the operations are different depending on whether the sound is voic^ or unvoiced. 

In the case of unvoiced sound, the synthesis demands a simple time shift (lengthening or shortening) of tiie aforesaid 

45 intervals on the basis of the ratio between the duration imposed by the prosodic rules and the intrinsic duration. In ttie 
case of voiced sound, instead, the method according to the invention is applied. 

The synthesis mettiod according to the invention starts from the consideration tiiat a voiced sound can be conskiered 
as a sequence of quasi-periodic intervals, each defined by a value Pa of the fundamental period. This is clearly seen in 
Figure 3. which shows the waveform of diphone "d.m". the related markers separating individual inten^als and. for each 

so inten^al, value Pa of tiie con'esponding period expressed in Hz. The part of Figure 3 between the two markers "v" cor- 
responds to the right portion of phoneme "§"; the part between tiie second marker V and the end-of-diphone marker 
T corresponds to tiie left part of phoneme "m". The aforesaid intervals may be considered as the inpulse responses 
of a filter, stationary for some milliseconds and con'esponding to the vocal duct, which is excited by a sequence of 
impulses synchronous witti tiie fundamental frequency of the source (vibrating frequency of tiie vocal cords). For each 

55 interval tiie synthesis module is to r^eive the original signal with fundamental period Pa (analysis period) and to provide 
a signal modified with period pg (synthesis period) require by prosodic rules. 

The essential information characterizing each speech interval is contained in tiie signal part Immediately following 
the excitation impulse (main part of tiie response), while ttie response itself becomes less and less significant as the 
distance from tiie inpulse position increases. Taking this into account, in the synthesis mettiod according to tiie invention 
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this main part is maintained as unchanged as possible and the lengthening or shortening of the period required by the 
prosodic rules are obtained by acting on the remaining part. 

For this purpose, an unchanging and a changeable part are then identified in each interval, and only the latter is 
involved in connection, overlap and add operations. The unchanging part of the original signal is not constant, but rather 

s it depends for each Interval on the ratio between Pg and Pa* This unchanging part lies between the start-of -i nterval marker 
and a so-called left analysis edge bga. which is one of the zero crossings of the original speech signal, identified with 
criteria that will be desaibed further on and that can be different depending on whether the synthesis period is longer, 
shorter or equal to the analysis period. The changeable part is delimits by the left analysis edge and by a so-called 
right analysis edge b^a* which essentially coincides with the end of the interval, In particular with the sample preceding 

10 the start-of-interval marker of the subsequent interval. 

In the synthesized signal, a left and a right synthesis edge b^, bds will con'espond to the left and right analysis edge 
bga. bc)a- For a given interval, the left synthesis edge obviously coincides with the left analysis ^ge, with reference to 
the start-of-interval marker, since the preceding part of signal is reproduced unaltered in the synthesis. The right syn- 
thesis edge is defined by relation 

15 

b<te = b33 + Ap (1) 

where Ap = p^ - p ^ will have a positive or negative value depending on whether, in synthesis, there is a lengthening or 
shortening of the fundamental period. 

20 The changeable part of the inten^al is modified by applying a pair of connecting functions whose duration is 
As = b ^ - b gg . The first function has a maximum value (specifically 1) in con^espondence with the left analysis edge 
and a minimum value (spectficaily 0) in correspondence with the point b^a + As. The second function has a maximum 
value (specifically 1) in con-espondence with the right analysis edge bd a a minimum value (specifically 0) in con'e- 
spondence with point b^a - As. The connecting functions can be of the kind commonly used for these purposes (e.g. 

25 Manning windows or similar functions). 

For the sake of further clarifying the invention. Figures 4 - 6 show some graphs illustrating the application of the 
method to af ictitious signal. In these Figures, part A shows three consecutive intervals of the original signal, with indexes 
i-1 . i. i+1 , and indicates also their fundamental periods p (h = 1-1 . 1, 1+1) as well as pitch (or start-of-interval) markers 
Ma and the left and right analysis edges bga. bda- Parts B and C show, for each interval, respectively the first and second 

30 connecting functions (which hereinafter shall be called for the sake of simplicity function B" and function C") and the 
time relations with the original signal. Part D shows the synthesized signal wavefbrn^ resulting from the method accord- 
ing to the invention, with the indication of the respective fundamental periods Psk (k = j-1 , j, j+1) . of pitch markers i\/is 
and of left and right synthesis edges b^, b^s* Part E is a representation of the waveform portion where, after the time 
shift, the waveforms obtained with the application of the two connecting functions to tiie changeable part of ttie original 

35 signal are submitted to the overlapping and adding process. Note tiiat tiie serial numbers of ttie intervals in analysis and 
synttiesis can be different, since suppressions or duplications of intervals may have oocunred previously. 

In particular, Figure 4 illustrates tiie case of an increase in fundamental period (and therefore decrease in frequency) 
in syntiiesis with respect to the original signal, in a signal portion where no interval suppressions or duplications have 
occurred. Weighting is carried out in each interval witii a respective pair of connecting functions. As a consequence of 

40 the period increase, duration As of the functions is greater tiian the lengtii of tiie variable part of the original signal, so 
that function B also interests tiie beginning of the waveform relate to the subsequent interval, while function C interests 
a part of waveform on the left of ttie left analysis edge. 

Figure 5 shows an analogous representation in the case of deaease in fundamental period (and ttierefore increase 
in frequency) in synthesis with respect to ttie original signal. In ttiis example too no interval suppressions or duplications 

45 occurred. In this case functions B, C interest a waveform portion with shorter duration than the portion tying between 
bsa and b^a- 

Finally. Figure 6 shows an example of increase in fundamental period in synthesis in the case of suppression of an 
interval of ttie original signal (tiie one with index i in ttie example). Two intervals are obtained in synthesis, indicated by 
indexes j-1 and j, which intervals respectively maintain, as unchanging part, the one of intervals with index i-1 and i+1 

50 in ttie original signal. The interval with index i+1 in ttie original signal is processed in the same way as each interval of 
original signal in Figure 4. The modified part of tiie interval with index j-1 in ttie synthesized signal, instead, is obtained 
by overlapping and adding the two waveforms obtained by weighting only witti function B the changeable part of ttie 
interval witti index i-1 in the original signal, and by weighting only witti function C the final part of ttie interval with index 
i in the original signal. In ottier words, function B is appli^ on the right of bga in the cun^ent interval to be reproduced in 

55 synthesis, and function C is applied on ttie left of the suk^sequent interval to be reproduced. These procedures of appli- 
cation of the connecting functions are quite general and are applied also in case of interval duplication and diphone 
change. 

Purely by way of example, for the diagrams in figures 4 - 6 ttie following functions were utilized: 
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0,5 - 0,5-cos{ JiI(A8-1 +b i)/(As-1 )] "} (function B) 

5 0.5 - 0.5-cos{7c[(x,-b88)/(As-1)]"} (function C) 

In these functions, beg. As have the meaning seen previously and are expressed as a number of samples; x,- is 
the generic sample of the variable part of the original waveform (with b3a^x,<bQg-i-As,f6r function B, and 
b^ - As ^ X, < b^g fbrfunction C);nisanumberwhichcan vary (e.g. from 1to3)dependingonratioAs/Pa:inparticular. 
10 in the drawing, n was considered to be 1 . Obviously, in the formulas, value 0.5 can be replaced by a generic value A/2 
if a function whose maximum is A instead of 1 is i^ed. or by a pair of values whose sum is 1 (or A). 

Figures 7A, 7B to 1 0A, 1 0B represent some real examples of application of the method, for two portions of diphone 
"d_m" of Figure 3, utilized in two different positions in the sentence where the synthesis rules require respectively a 
decrease and an increase in fundamental period (and therefore an increase and respectively a decrease in fundamental 
15 frequency). For all intervals, pitch markers, left analysis and synthesis edges and fundamental frequency, both in analysis 
and synthesis, are indicated. Figures with letter A show the original waveform and Rgures with letter B the synthesized 
signal. Figures 7A, 7B. 8A, 8B show the first two intervals of the diphone being examined (phoneme "^") in case of 
increase (Figures 7A, 78) and respectively of decrease (Figures 8A, SB) of the fundamental frequency. Figures 9A, 98, 
10A, 10B show instead the first two inten/als of phoneme "m** in the same conditions as illustrated in Figures 7, 8. As 
20 an effect of the frequency decrease, only the first interval is completely visible in Figures 8B and 1 0B. 

A preferred embodiment of the method adopted to identify the left analysis and synthesis edge for each interval to 
be reproduced in synthesis will now be described. In the example described, a different method is used depending on 
whether the fundamental period in synthesis is smaller than or equal to the period in analysis, or it is greater. 
Figure 1 1 is the general flow chart of the operations caried out if Ps ^ Pa- 
25 The first operation is the computation of function ZCR (Zero Crossing Rate) indicating the number of zero crossings 
(step 11). In this computation, zero crossings that are spaced apart from the previous one by less tiian a limited number 
of signal samples (e.g. 10) are neglected, in order to eliminate non-significant oscillations of the signal. 

As can be seen in Figure 13, tiie zero aossings tiiat are considered are assign^ an index varying from 1 to tiie 
desaiptor of tiie total zero crossing number LZV (step 110). Moreover, ttie following variables are assigned (step 111): 

30 

• (right analysis edge) to the value of analysis period Pai 

- bds (right synthesis edge) to the value of synthesis period b^a + Ap; 

Diff_a_s to tiie absolute value |Ap| of the difference between the analysis and synthesis periods. 

35 In these relations, as in ttiose examine further on. tiie values of the periods and the lengths of certain intervals are 
expressed in terms of number of samples. 

Going back to Figure 1 1 , after computing function ZCR, a check is made (st^ 1 2) tiiat the number of zero crossings 
found in step 1 1 is not lower tiian a minimal threshold of zero crossings lndZ__i\/iin (e.g. 5 crossings). Actually, according 
to the invention, it is desired to r^roduce unaltered, in tiie syntiiesized signal, the oscillations immediately following the 

40 excitation impulse, which oscillations, as stated, are the most significant ones. If the check yields a positive result, a 
possible candidate is searched among tiie zero crossings tiiat were found (step 13) and subsequentiy a first phase of 
search for the left syntiiesis and analysis ^ges bgs, bga is carried out (step 14). If at tiie end of step 14 no suitable zero 
crossing has been found, a search continuation phase is started (step 15) and, if after tills phase tiie left syntiiesis and 
analysis edges have not yet been identified, tiien a phase of continuation and conclusion of tiie search is started (step 

45 1 7). If tiie comparison in step 1 2 indicates tiiat the number of zero crossings is lower than the threshold, tiien tiie zero 
crossing witii index J » lndZ_Min is arbitrarily considered as a candidate (step 18) and a search for bga and bgs (step 
19), identical to tiie one carried out in step 14, is performed: if this search is unsuccessful, then step 17, i.e. the search 
continuation and conclusion, is directly started, witiiout going through step 15, for reasons that will become clear when 
the latter is desaibed. 

50 A step analogous to step 1 7 is envisaged also in case of lengtiiening of flie fundamental period in synthesis, as will 
be seen furtiier on. For tiie sake of simplicity, tiie same flow chart was used for both cases, which are distinguish^ by 
means of some conditions of entry into the step itself. In particular, for the case p^ ^ Pa tiie conditions r_P ^ 1 (where 
r_P is the ratio PsfPeif Start = 0. End - LZV , Step ^ +1 (step 1 6 in Figure 1 1 ) are set. The first condition is evident. The 
other tiiree indicate that the cycle of examination of tiie zero crossings envisaged in phase 1 7 will be carried out in tiie 

55 order of Increasing indexes. 

The operations performed in steps 13-15 and 1 7 will be described in detail furtiier on, witti reference to Figures 14 
-17. 

Figure 12 Is the general flow chart of the operations carried out if tiie synthesis period Pa is longer than the analysis 
period Pa. The first operation (step 21) consists again in computing function ZCR and is identical to step 1 1 in Figure 



EP0 706170 A2 



1 1 . Subsequently (step 22) a search is carried out tor the left synthesis and analysis edges, with procedures that will be 
described with reference to Figure 18. and, if this phase does not have a positive outcome, a search continuation and 
conclusion phase is initiated (step 24), con^esponding to st^ 1 7 in Figure 1 1 . Conditions r_P > 1 . Start = LZV- 1 , End 
- -1 , Step = -1 are set for the operations envisaged in step 24. The first condition is evident. The other three indicate 
5 that the cycle of examination of the zero crossings envisaged in step 24 will be carri^ out in this case in the order of 
decreasing indexes. 

Figure 14 shows the flow chart of the search for a zero crossing which is candidate to act as left analysis and 
synthesis edge (step 13 in Figure 1 1). J denotes the index of the candidate. In particular, the central zero crossing, 
whose index is J = (LZV+1)/2 (stq3 130). is initially examined as a candidate and its abscissa ZCR(J) is compared with 

10 the right synthesis edge bds (step 1 3 1 ) . If this initial candidate is already on the left of the right synthesis edge, the phase 
of search for the left analysis and synthesis edge (step 14, Figure 11) is started directly In the opposite case, zero 
crossings on the left of the central one are examined with a backwards cycle, searching for a candidate whose abscissa 
is on the left of bds (steps 132-134). When a zero aossing that meets this condition is found, it is considered as a 
candidate (step 135) and the search phase (step 14 in Figure 1) is starts after verifying that the index of the candidate 

75 is not (LZV+1)/2 (step 136). In effect, a backward search cycle has been performed because the initial candidate, with 
index (LZV+1 )/2, was on the right of b^^* and hence obtaining a candidate with that index signals an anomalous condition: 
if this occurs, the search phase is started after setting J=0. The same operations are performed if the cycle ends before 
a candidate is found. 

Figure 15 shows the operations can^i^ out for the first phase of search for bge, t^ea i^^P 1^ '^^ Figure 1 1). For this 

20 search, a backward examination is made of the zero crossings starting from the one preceding LZV, and the distance 
Diff__z_a between the right analysis edge b^a and the current zero crossing ZCR(i) is calculated (st^s 140. 141). This 
distance, multiplied by r_P (ratio between the synthesis period pg and the analysis period pj is compared with Diff_a_s 
(step 142). to check that there is a time interval sufficient to apply the connecting function. Weighting with r_P links the 
duration of that function to the percentage shortening of the period and it is aimed at guaranteeing a good connection 

25 between subsequent intervals. If Diff_a_s > Diff_z_a*r_P , the search cycle continues (step 143), until a zero crossing 
is found such that Diff_a_s ^ (Diff.z_a*r_P) or until all zero crossings have been considered: in the latter case step 14 
is ended and step IS (Figure 1 1) of search continuation, is started. When the condition Diff_a_s ^ Diff_z_a*r_P is met, 
the current index i is compared with index J of the candidate (step 144). If i < J, the cycle is continue. If the two indexes 
are equal, then the current zero crossing is considered as left analysis edge bga and as left synthesis ^ge (step 

30 1 47); if instead i > J, then distance A_a between the right analysis edge bda and the current zero crossing ZCR(i). distance 
A_s between the right synthesis ^ge b^s and the current zero crossing ZCR(i). and ratio A between A_s and A_a are 
calculated (step 145). and ratio A is compared to the value (r_P)/2 (step 146). If A ^ (r_P)/2 , then the tasks of left 
analysis edge b^a and left synthesis edge b^s are assigned to the current zero crossing (step 147). othenvise phase 15 
(Figure 1 1) of search continuation is started. The last comparison indicates that not only a sufficient distance between 

35 the left and right synthesis edge is r^uir^, but also that the connecting function takes into account the shortening in 
synthesis; this, too, helps obtaining a good connection between adjacent intervals. 

Variable TRUE" in the last step 1 47 in Figure 1 5 indicates that bga and bgg have been found and disables subsequent 
search phases. The same variable will also be utilized with the same meaning in the other flow charts related to the 
search for the left analysis and synthesis edges. 

40 St^> 14 allows finding a candidate, if any. that lies on the left of the right synthesis edge and is as close as possible 
to it. while guaranteeing a time interval sufficient to apply the connecting function; this step is the core of the criterion 
of the search for b^ and beg. 

Search continuation st^ 15 is illustrated in detail in Figure 16. This step, if it is pertormed (negative result of phase 
14 and theretore of tiie check on the TRUE condition in step 150). starts with a new comparison between LZV and 

45 lndZ_min (step 151), aimed now at just verifying whether LZV > lndZ_min. If the condition is not met, then step 17. of 
search continuation and conclusion is initiated. If LZV > IndZ.Min, then a check is made on whetiier tiie zero crossing 
having index lndZ_Min is positioned on the left of the right synthesis edge b^s (st^ 1 52) . In tiie affirmative, tiiis crossing 
is considered to be the left analysis edge bga and left synthesis edge bgs (step 153). If instead tiie zero crossing having 
index lndZ_Mln is still on tiie right of the right syntiiesis ^ge. then step 17 (Figure 11) of search continuation and 

50 conclusion is initiated. 

Search continuation and conclusion step 1 7 is r^resented in detail in Figure 1 7. After ch^ng the need to perform 
it (step 1 70). ttie zero crossings are review^ again, in increasing index order. In ttie examination cycle (steps 1 71 - 1 74 
In Figure 1 7), a check Is made at each step on whether tiie current zero crossing (indicated by Z.Tmp) is on the left of 
the right synthesis ^ge bds end its distance from such edge is not lower than a predetermined minimum value 5. e.g. 
55 10 signal samples (step 173). If the two conditions are not met. tiien tiie subsequent zero crossing is examined (step 
174). ottienfvise tiiis zero crossing is temporarily considered as the left synthesis and analysis edge (step 1 75) and tiie 
cyde is continue. The last zero crossing that meets condition 1 73 will be considered as the left synthesis and analysis 
^ge (step 1 79). The check on r_P at step 1 76 is an additional means to distinguish between tiie case Ps ^ Pa and tiie 
case Ps > Pa, and it causes steps 1 77 and 1 78 of the flow chart to be omitted in the case being examined. 
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Figure 18 illiistrates the search for and when the synthesis period is lengthened with respect to the analysis 
period. This search starts with a conparison between the lengthening in synthesis Diff_a_s and half the duration of the 
analysis period pa (st^ 220). If Diff_a_s > Pa/2. step 24 (illustrated in detail in Figure 17) is started directly If 
Oiff_a_s ^ Pa/2 , a backward search cyde is carried out. starting from the zero crossing preceding LZV. Distance 

5 Diff_z_a between the right analysis edge bda and the current zero crossing ZCR(i) (steps 221 . 222) is calculated and is 
compared with Diff_a.s (step 223): if it is smaller, then the search cycle continues (step 224), othenA/ise the current zero 
crossing is considered as the left analysis and synthesis ^ge (step 225). If, at the end of the cycle, b^a and b^s have 
not been determined, then the phase of search continuation and conclusion is initiated (phase 24, Figure 12). 

If the lengthening r^uired in synthesis Is less than or equal to half the analysis period, the operations described 

10 above allow finding a candidate, if any, that is the first for which the distance from the right analysis edge exceeds or is 
equal to the required lengthening. 

In the search continuation and conclusion phase, a backward search cycle is carried out, as stated, starting from 
the zero crossing preceding LZV. with the procedures illustrated in steps 171 - 175 in Figure 17. l\^oreover, since a 
lengthening of the interval is cor^idered (step 176). distance A_a between the right analysis edge b^a and the current 

15 zero crossing Z_Tmp, distance A_s between the right synthesis edge b^s and the current zero crossing Z_Tmp and ratio 
A between these distances are computed (step 1 77) for the zero crossings that meet the conditions of step 1 73. Ratio 
A is compared with twice the ratio between the periods (r_P*2) for the same reasons seen for comparison 1 46 in Figure 
15, and the zero crossing that meets the condition A ^ (r_P^2) will be taken as left analysis edge and left synthesis 
edge b^s- 

20 The conditions impost in this phase allow assigning the task of left analysis edge to a zero crossing that lies on 
the left of the right synthesis ^ge, is as close as possible to it and also guarantees a sufficient time interval for the 
conn^ing function to be applied: in particular, given a certain analysis period, a left analysis edge positioned forther 
back in the original period will correspond to a greater lengthening required in synthesis. 

The method described herein can be performed by means of a conventional personal computer, workstation, or 

25 similar apparatus. 

It is evident that what is described above is given by way of non-limiting example and that variations and modifications 
are possible without departing from the scope of the invention. 

Claims 

30 

1 . Method for speech signal synthesis by means of time concatenation of waveforms representing elementary speo^h 
signal units, in which: at least the waveforms associated to voiced sounds are subdivided into a plurality of intervals, 
con'e^nding to the responses of the vocal duct to a series of impulses of vocal cord excitation, synchronous with 
the fundamental fr^uency of the signal; the waveform in each inten/al is weighted; the signals resulting from the 
35 weighting are replaced with a replica thereof shifted in time by an amount d^ending on a prosodic information ; and 
synthesis is performed by overlapping and adding the shifted signals; 
characterized in that: 

a cun-ent interval of original signal to be reproduced in synthesis is subdivided into an unchanging part, which 
40 lies between the inten^al beginning and a left analysis edge represented by a zero crossing of the original spe^h 

signal which meets predetermined conditions, and a variable part, which lies between the left analysis edge 
and a right analysis edge that essentially coincides with the end of the current interval, the left and right analysis 
edges being associate, in tiie synthesized signal, respectively with a left synttiesis edge and a right synthesis 
edge, of which the former coincides with the left analysis edge, with reference to a start-of-interval marker, and 
45 the latter coincides essentially with the end of the interval in the synthesize signal; 

- a first connecting function is applied on the part of waveform on tiie right of the left analysis edge of the current 
interval of original signal, which function has a duration equal to that of the segment of synthesized waveform 
lying between the left and right synthesis edges and an amplitude that progressively decreases and is maximum 
in correspondence with tiie left analysis edge; 

50 • a second connecting function is applied on the part of waveform on tiie left of tiie subsequent interval of original 
signal to be reproduced in synthesis, which function has a duration equal to that of tiie segment of synthesized 
waveform lying between the left and right syntiiesis edges and an amplitude tiiat progressively increases and 
is maximum in correspondence with tiie beginning of said subsequent interval; 

- each interval of syntiiesiz^ signal is built by reproducing unchanged the waveform in tiie unchanging part of 
55 tiie original interval and by joining ttiereto tiie waveform obtain^ by aligning in time and adding the two wave- 
forms resulting from applying tiie two connecting functions. 



i 



EP0706170 A2 



2. Methodaccording to claim 1 , characterized in that, if the duration of an interval is reduced or maintained unchanged 
for the synthesis with respect to the duration of the con'esponding interval of the original signal, the left analysis 
edge and the left synthesis edge are determined with the following operations: 

5 • computing the number of zero crossings of the original signal waveform and assigning each zero crossing an 
index, increasing from the beginning towards the end of the interval; 

- checking that the number of zero crossings is not lower than a first threshold; 

- searching, in case of positive outcome of the checK for a zero crossing candidate to act as left analysis and 
synthesis edge; 

10 - backwards searching, among all zero crossings in the interval, except the last one. for a candidate that lies on 
the left of the right synthesis ^ge, is as close as possible to it and guarantees a time interval sufficient for the 
connecting functions to be applied, and assigning the task of left analysis and synthesis edge to this candidate. 

3. i\^ethod according to claim 2, characterized in that, in said computation of the zero crossings, zero crossings whose 
75 distance from the previous one is lower than a predetermined distance are not taken into consideration. 

4. Method according to claim 2 or 3, characterize in that, if the backwards search has yielded a negative result and 
if the number of zero crossings is higher than the first threshold, the tasks of left analysis edge and left synthesis 
edge are assign^ to the zero crossing whose index con'esponds to said threshold, if such a zero aossing lies on 

20 the left of the right synthesis edge. 

5. Method according to any of claims 2 to 4. characterized in that if the backwards search has yielded a negative result 
and if the number of zero aossings is not higher than the first threshold, a further search phase is carried out to 
identify the zero crossings lying on the left of the right synthesis edge and having a distance from the latter that is 

25 not lower than a second threshold, and the tasks of left analysis edge and right analysis edge are assigned to the 
highest index zero crossing which meets these conditions. 

6. Method according to any of claims 2 to 5, characterize in that, if the comparison with the first threshold Indicates 
that the number of zero crossings is lower than the first threshold, said backwards search is performed directly and. 

30 if it yields negative a result, said further search phase is performed directly. 

7. Method according to any of claims 1 to 6, characterized in that, if the duration of the interval is increased for the 
synthesis conrpared to the duration of the corresponding interval of the original signal, the left analysis edge and 
the right synthesis edge are determined with the following operations: 

35 

computing the number of zero crossings of the original signal waveform; 

- comparing the duration lengthening of the synthesis interval and the duration of the original interval, to check 
that the lengthening does not exceed half the original Interval duration; 

if the check yieMs a positive result, searching backwards, among all the zero crossings except the last one, for 
40 a candidate zero crossing that lies on the left of the right synthesis edge and Is the first for which the distance 

from the right synthesis edge is not shorter than the lengthening of the interval duration, the tasks of left analysis 
edge and left synthesis edge being assigned to the zero crossing that meets said condition, if any. 

8. Method according to claim 7, characterized in that, in said computation of the zero crossings, the crossings whose 
45 distance from the previous crossing is lower than a predetermined distance are not taken into consideration. 

9. Method according to daim 7 or 8. characterized in that, if the inten^l duration lengthening exceeds half the original 
Interval duration or if the backwards search is unsuccessful, a further backwards search phase is earned out to 
identify the zero crossings lying on the left of the right synthesis edge and having a distance from the latter that is 

50 not lower than a third threshoM; the distances from the right synthesis edge and from the right analysis edge and 
the ratio between these distances are computed for such zero crossings; said ratio is compared with the value of 
the ratio between the duration of the synthesis interval and the duration of the original interval, and the tasks of left 
analysis edge and left synthesis edge are assigned to the zero crossing whose index is the lowest among those for 
which the ratio between the distances from the edges does not exceed by more than a predetermined bctor the 

55 ratio between durations. 
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